Optimal search strategies for identifying sound clinical prediction studies in EMBASE

BACKGROUND: Clinical prediction guides assist clinicians by pointing to specific elements of the patient's clinical presentation that should be considered when forming a diagnosis, prognosis or judgment regarding treatment outcome. The numbers of validated clinical prediction guides are growing in the medical literature, but their retrieval from large biomedical databases remains problematic and this presents a barrier to their uptake in medical practice. We undertook the systematic development of search strategies ("hedges") for retrieval of empirically tested clinical prediction guides from EMBASE. METHODS: An analytic survey was conducted, testing the retrieval performance of search strategies run in EMBASE against the gold standard of hand searching, using a sample of all 27,769 articles identified in 55 journals for the 2000 publishing year. All articles were categorized as original studies, review articles, general papers, or case reports. The original and review articles were then tagged as 'pass' or 'fail' for methodologic rigor in the areas of clinical prediction guides and other clinical topics. Search terms that depicted clinical prediction guides were selected from a pool of index terms and text words gathered in house and through request to clinicians, librarians and professional searchers. A total of 36,232 search strategies composed of single and multiple term phrases were trialed for retrieval of clinical prediction studies. The sensitivity, specificity, precision, and accuracy of search strategies were calculated to identify which were the best. RESULTS: 163 clinical prediction studies were identified, of which 69 (42.3%) passed criteria for scientific merit. A 3-term strategy optimized sensitivity at 91.3% and specificity at 90.2%. Higher sensitivity (97.1%) was reached with a different 3-term strategy, but with a 16% drop in specificity. The best measure of specificity (98.8%) was found in a 2-term strategy, but with a considerable fall in sensitivity to 60.9%. All single term strategies performed less well than 2- and 3-term strategies. CONCLUSION: The retrieval of sound clinical prediction studies from EMBASE is supported by several search strategies

Could the clinical interpretability of subgroups detected using clustering methods be improved by using a novel two-stage approach?

Background: Recognition of homogeneous subgroups of patients can usefully improve prediction of their outcomes and the targeting of treatment. There are a number of research approaches that have been used to recognise homogeneity in such subgroups and to test their implications. One approach is to use statistical clustering techniques, such as Cluster Analysis or Latent Class Analysis, to detect latent relationships between patient characteristics. Influential patient characteristics can come from diverse domains of health, such as pain, activity limitation, physical impairment, social role participation, psychological factors, biomarkers and imaging. However, such 'whole person' research may result in data-driven subgroups that are complex, difficult to interpret and challenging to recognise clinically. This paper describes a novel approach to applying statistical clustering techniques that may improve the clinical interpretability of derived subgroups and reduce sample size requirements. Methods: This approach involves clustering in two sequential stages. The first stage involves clustering within health domains and therefore requires creating as many clustering models as there are health domains in the available data. This first stage produces scoring patterns within each domain. The second stage involves clustering using the scoring patterns from each health domain (from the first stage) to identify subgroups across all domains. We illustrate this using chest pain data from the baseline presentation of 580 patients. Results: The new two-stage clustering resulted in two subgroups that approximated the classic textbook descriptions of musculoskeletal chest pain and atypical angina chest pain. The traditional single-stage clustering resulted in five clusters that were also clinically recognisable but displayed less distinct differences. Conclusions: In this paper, a new approach to using clustering techniques to identify clinically useful subgroups of patients is suggested. Research designs, statistical methods and outcome metrics suitable for performing that testing are also described. This approach has potential benefits but requires broad testing, in multiple patient samples, to determine its clinical value. The usefulness of the approach is likely to be context-specific, depending on the characteristics of the available data and the research question being asked of it

Prediction of persistent shoulder pain in general practice: Comparing clinical consensus from a Delphi procedure with a statistical scoring system

<p>Abstract</p> <p>Background</p> <p>In prognostic research, prediction rules are generally statistically derived. However the composition and performance of these statistical models may strongly depend on the characteristics of the derivation sample. The purpose of this study was to establish consensus among clinicians and experts on key predictors for persistent shoulder pain three months after initial consultation in primary care and assess the predictive performance of a model based on clinical expertise compared to a statistically derived model.</p> <p>Methods</p> <p>A Delphi poll involving 3 rounds of data collection was used to reach consensus among health care professionals involved in the assessment and management of shoulder pain.</p> <p>Results</p> <p>Predictors selected by the expert panel were: symptom duration, pain catastrophizing, symptom history, fear-avoidance beliefs, coexisting neck pain, severity of shoulder disability, multisite pain, age, shoulder pain intensity and illness perceptions. When tested in a sample of 587 primary care patients consulting with shoulder pain the predictive performance of the two prognostic models based on clinical expertise were lower compared to that of a statistically derived model (Area Under the Curve, AUC, expert-based dichotomous predictors 0.656, expert-based continuous predictors 0.679 vs. 0.702 statistical model).</p> <p>Conclusions</p> <p>The three models were different in terms of composition, but all confirmed the prognostic importance of symptom duration, baseline level of shoulder disability and multisite pain. External validation in other populations of shoulder pain patients should confirm whether statistically derived models indeed perform better compared to models based on clinical expertise.</p

Comparison of the effectiveness of three manual physical therapy techniques in a subgroup of patients with low back pain who satisfy a clinical prediction rule: Study protocol of a randomized clinical trial [NCT00257998]

BACKGROUND: Recently a clinical prediction rule (CPR) has been developed and validated that accurately identifies patients with low back pain (LBP) that are likely to benefit from a lumbo-pelvic thrust manipulation. The studies that developed and validated the rule used the identical manipulation procedure. However, recent evidence suggests that different manual therapy techniques may result similar outcomes. The purpose of this study is to investigate the effectiveness of three different manual therapy techniques in a subgroup of patient with low back pain that satisfy the CPR. METHODS/DESIGN: Consecutive patients with LBP referred to physical therapy clinics in one of four geographical locations who satisfy the CPR will be invited to participate in this randomized clinical trial. Subjects who agree to participate will undergo a standard evaluation and complete a number of patient self-report questionnaires including the Oswestry Disability Index (OSW), which will serve as the primary outcome measure. Following the baseline examination patients will be randomly assigned to receive the lumbopelvic manipulation used in the development of the CPR, an alternative lumbar manipulation technique, or non-thrust lumbar mobilization technique for the first 2 visits. Beginning on visit 3, all 3 groups will receive an identical standard exercise program for 3 visits (visits 3,4,5). Outcomes of interest will be captured by a therapist blind to group assignment at 1 week (3(rd )visit), 4 weeks (6(th )visit) and at a 6-month follow-up. The primary aim of the study will be tested with analysis of variance (ANOVA) using the change in OSW score from baseline to 4-weeks (OSW(Baseline )– OSW(4-weeks)) as the dependent variable. The independent variable will be treatment with three levels (lumbo-pelvic manipulation, alternative lumbar manipulation, lumbar mobilization). DISCUSSION: This trial will be the first to investigate the effectiveness of various manual therapy techniques for patients with LBP who satisfy a CPR

Clinical Prediction Rule for Stratifying Risk of Pulmonary Multidrug-Resistant Tuberculosis

Multidrug-resistant tuberculosis (MDR-TB), resistance to at least isoniazid and rifampin, is a worldwide problem.To develop a clinical prediction rule to stratify risk for MDR-TB among patients with pulmonary tuberculosis.Derivation and internal validation of the rule among adult patients prospectively recruited from 37 health centers (Perú), either a) presenting with a positive acid-fast bacillus smear, or b) had failed therapy or had a relapse within the first 12 months.Among 964 patients, 82 had MDR-TB (prevalence, 8.5%). Variables included were MDR-TB contact within the family, previous tuberculosis, cavitary radiologic pattern, and abnormal lung exam. The area under the receiver-operating curve (AUROC) was 0.76. Selecting a cut-off score of one or greater resulted in a sensitivity of 72.6%, specificity of 62.8%, likelihood ratio (LR) positive of 1.95, and LR negative of 0.44. Similarly, selecting a cut-off score of two or greater resulted in a sensitivity of 60.8%, specificity of 87.5%, LR positive of 4.85, and LR negative of 0.45. Finally, selecting a cut-off score of three or greater resulted in a sensitivity of 45.1%, specificity of 95.3%, LR positive of 9.56, and LR negative of 0.58.A simple clinical prediction rule at presentation can stratify risk for MDR-TB. If further validated, the rule could be used for management decisions in resource-limited areas

Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews

Background: The interest in prognostic reviews is increasing, but to properly review existing evidence an accurate search filer for finding prediction research is needed. The aim of this paper was to validate and update two previously introduced search filters for finding prediction research in Medline: the Ingui filter and the Haynes Broad filter. Methodology/Principal Findings: Based on a hand search of 6 general journals in 2008 we constructed two sets of papers. Set 1 consisted of prediction research papers (n = 71), and set 2 consisted of the remaining papers (n = 1133). Both search filters were validated in two ways, using diagnostic accuracy measures as performance measures. First, we compared studies in set 1 (reference) with studies retrieved by the search strategies as applied in Medline. Second, we compared studies from 4 published systematic reviews (reference) with studies retrieved by the search filter as applied in Medline. Next -using word frequency methods - we constructed an additional search string for finding prediction research. Both search filters were good in identifying clinical prediction models: sensitivity ranged from 0.94 to 1.0 using our hand search as reference, and 0.78 to 0.89 using the systematic reviews as reference. This latter performance measure even increased to around 0.95 (range 0.90 to 0.97) when either search filter was combined with the additional string that we developed. Retrieval rate of explorative prediction research was poor, both using our hand search or our systematic review as reference, and even combined with our additional search string: sensitivity ranged from 0.44 to 0.85. Conclusions/Significance: Explorative prediction research is difficult to find in Medline, using any of the currently available search filters. Yet, application of either the Ingui filter or the Haynes broad filter results in a very low number missed clinical prediction model studie

Prediction of High-Grade Vesicoureteral Reflux after Pediatric Urinary Tract Infection: External Validation Study of Procalcitonin-Based Decision Rule

BACKGROUND: Predicting vesico-ureteral reflux (VUR) 653 at the time of the first urinary tract infection (UTI) would make it possible to restrict cystography to high-risk children. We previously derived the following clinical decision rule for that purpose: cystography should be performed in cases with ureteral dilation and a serum procalcitonin level 650.17 ng/mL, or without ureteral dilatation when the serum procalcitonin level 650.63 ng/mL. The rule yielded a 86% sensitivity with a 46% specificity. We aimed to test its reproducibility. STUDY DESIGN: A secondary analysis of prospective series of children with a first UTI. The rule was applied, and predictive ability was calculated. RESULTS: The study included 413 patients (157 boys, VUR 653 in 11%) from eight centers in five countries. The rule offered a 46% specificity (95% CI, 41-52), not different from the one in the derivation study. However, the sensitivity significantly decreased to 64% (95%CI, 50-76), leading to a difference of 20% (95%CI, 17-36). In all, 16 (34%) patients among the 47 with VUR 653 were misdiagnosed by the rule. This lack of reproducibility might result primarily from a difference between derivation and validation populations regarding inflammatory parameters (CRP, PCT); the validation set samples may have been collected earlier than for the derivation one. CONCLUSIONS: The rule built to predict VUR 653 had a stable specificity (ie. 46%), but a decreased sensitivity (ie. 64%) because of the time variability of PCT measurement. Some refinement may be warranted

Research methods for subgrouping low back pain

<p>Abstract</p> <p>Background</p> <p>There is considerable clinician and researcher interest in whether the outcomes for patients with low back pain, and the efficiency of the health systems that treat them, can be improved by 'subgrouping research'. Subgrouping research seeks to identify subgroups of people who have clinically important distinctions in their treatment needs or prognoses. Due to a proliferation of research methods and variability in how subgrouping results are interpreted, it is timely to open discussion regarding a conceptual framework for the research designs and statistical methods available for subgrouping studies (a method framework). The aims of this debate article are: (1) to present a method framework to inform the design and evaluation of subgrouping research in low back pain, (2) to describe method options when investigating prognostic effects or subgroup treatment effects, and (3) to discuss the strengths and limitations of research methods suitable for the hypothesis-setting phase of subgroup studies.</p> <p>Discussion</p> <p>The proposed method framework proposes six phases for studies of subgroups: studies of assessment methods, hypothesis-setting studies, hypothesis-testing studies, narrow validation studies, broad validation studies, and impact analysis studies. This framework extends and relabels a classification system previously proposed by McGinn et al (2000) as suitable for studies of clinical prediction rules. This extended classification, and its descriptive terms, explicitly anchor research findings to the type of evidence each provides. The inclusive nature of the framework invites appropriate consideration of the results of diverse research designs. Method pathways are described for studies designed to test and quantify prognostic effects or subgroup treatment effects, and examples are discussed. The proposed method framework is presented as a roadmap for conversation amongst researchers and clinicians who plan, stage and perform subgrouping research.</p> <p>Summary</p> <p>This article proposes a research method framework for studies of subgroups in low back pain. Research designs and statistical methods appropriate for sequential phases in this research are discussed, with an emphasis on those suitable for hypothesis-setting studies of subgroups of people seeking care.</p